Skip to content

fix(extensions): stop deleting user files when removing supplemental skills - #5467

Open
aarjang wants to merge 3 commits into
github:mainfrom
aarjang:fix/5381-skill-removal-rmtree
Open

aarjang wants to merge 3 commits into
github:mainfrom
aarjang:fix/5381-skill-removal-rmtree

Conversation

@aarjang

@aarjang aarjang commented Nov 17, 2026

Copy link
Copy Markdown

Fixes #5381.

Problem

ExtensionManager._unregister_extension_skills removes a non-generic supplemental extension skill directory with shutil.rmtree(skill_subdir). Ownership of such a directory is established through the SKILL.md provenance marker alone (metadata.source: extension:<id>), not per file — so any file a user added next to it (notes, extra references) is deleted on extension remove, and extension update runs the same removal before reinstalling. The generic branch of the same method already removes only its hash-verified SKILL.md and then rmdirs, keeping unrelated files.

Reproduced with the issue's scenario (supplemental skills route): install an extension in skills mode, add my-notes.md next to the generated SKILL.md, run extension remove — the user's notes are gone.

Fix

The removal fallback now applies the same per-file discipline as the generic branch: unlink the extension's own SKILL.md (ownership is already marker-verified by _find_extension_skill_dirs), then rmdir the directory only once it is empty. A directory holding user files survives with those files intact; an emptied directory is still cleaned up. The fallback also adopts the generic branch's symlink rule: a SKILL.md symlink the user re-pointed outside this extension's installed directory is not our rendered output and is left alone, while a dev-mode link into a .specify-dev/extension-skills cache stays removable (pinned by the existing test_generic_update_rollback_preserves_cross_extension_skill_link).

Tests & evidence

  • test_remove_preserves_user_files_in_extension_skill_dir — installs on the supplemental route, adds a user file beside the generated SKILL.md, removes the extension: the extension's SKILL.md is gone, the user file survives. Fails on main (the user file is deleted by rmtree), passes on this branch.
  • test_remove_still_deletes_emptied_skill_dir — with no user files, the directory itself is still removed (pins the rmdir cleanup); passes on both, kept as a contract.
  • tests/test_extension_skills.py plus the full tests/specify_cli/extensions and tests/specify_cli/integrations suites pass locally (.venv/bin/python -m pytest per CONTRIBUTING).
  • Full suite: 16,631 passed, 594 skipped, 0 failed on this branch (macOS, .venv/bin/python -m pytest).

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Written with Claude Code (model: Claude Fable 5, default reasoning effort, human-supervised). The agent reproduced the data-loss behavior from the issue, wrote the fix mirroring the generic branch's per-file removal, and ran the verification; I reviewed the diff before submitting. The commit carries a matching Assisted-by: trailer.

🤖 Generated with Claude Code

@aarjang
aarjang requested a review from mnriem as a code owner November 17, 2026 19:41
Copilot AI adaptive review requested due to automatic review settings November 17, 2026 19:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Supplemental skill updates can silently leave SKILL.md missing when user notes are preserved, and the new symlink-removal fallback lacks direct positive and negative test coverage.

2 open issues
What changed in this PR

Addresses #5381 by changing Specify CLI’s supplemental extension-skill cleanup to preserve user files beside SKILL.md.

Changes:

  • Replaces recursive directory deletion with SKILL.md removal and empty-directory cleanup.
  • Adds symlink handling to preserve externally repointed links while allowing dev-cache link removal.
  • Adds regression tests for preserving user notes and removing emptied skill directories.

Review generated by GitHub Copilot for the requester (model unknown; automated mode). Tests were not run during this review.

File Description
tests/​test_extension_skills.py Adds user-file preservation and empty-directory cleanup tests.
src/​specify_cli/​extensions/​__init__.py Limits supplemental skill removal to owned output and adds symlink safeguards.

Review effort: Balanced (adaptive)

Comment thread src/specify_cli/extensions/__init__.py Resolved
Comment thread src/specify_cli/extensions/__init__.py Resolved
@mnriem mnriem added the triage-must-have Verdict: high-value, important work for Spec Kit — do first label Nov 17, 2026
Copilot AI adaptive review requested due to automatic review settings November 17, 2026 20:04
@aarjang

aarjang commented Nov 17, 2026

Copy link
Copy Markdown
Author

Addressed both findings in 762e200:

  • Replacement install vs. a directory kept alive by user files — _unregister_extension_skills now returns the directories whose ownership it verified before removing their SKILL.md; remove() keeps them on the manager, and the force-reinstall (update) path hands them to _register_extension_skills as recreatable_dirs. A pre-existing directory in that list may receive a fresh SKILL.md even though it now lacks a marker — the verification happened moments earlier in the same replacement operation — while the guard for unrelated user directories is unchanged. test_force_reinstall_recreates_skill_beside_user_notes pins the end-to-end flow on the supplemental route: install, add my-notes.md, force-reinstall — the refreshed SKILL.md and the notes both survive (fails before this commit: the skill silently disappeared).
  • Supplemental-route symlink coverage — added the positive/negative pair exercising the fallback without generic hashes: test_fallback_preserves_user_retargeted_skill_symlink (a marker-matching link re-pointed at another extension's file survives, with its directory) and test_fallback_removes_dev_cache_symlink_outside_installed_dir (an expected .specify-dev/extension-skills link whose source tree lives outside the installed directory is retired and the emptied directory cleaned up). Both fail against the previous commit.

tests/test_extension_skills.py plus the extensions, integrations, and generic-integration suites pass locally (1,906 tests).

AI disclosure: This comment and the commit were drafted with Claude Code (model: Claude Fable 5), human-directed and reviewed, on behalf of @aarjang.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Integration switching and extension updates can leave preserved skill directories without SKILL.md or registry entries because verified ownership context is lost between removal and re-registration.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Medium severity Restore extension skills when switching away and back

src/​specify_cli/​extensions/​__init__.py:1671

Switching away from an integration calls unregister_agent_artifacts(), which now preserves user notes but removes SKILL.md and discards the verified directory paths. Switching back calls register_enabled_extensions_for_agent() without recreation context, so this guard skips the surviving directory and the extension skill does not return. Preserve safe ownership evidence across these separate lifecycle operations, and add a switch-away/switch-back regression that checks skill recreation and preservation of the notes.

AI-generated review for the requesting contributor by GitHub Copilot (model: unknown; automated review mode).

Medium severity Recreate skills when updating extensions with user notes

src/​specify_cli/​extensions/​__init__.py:2601

The actual extension update transaction calls remove(..., keep_config=True) and then install_from_zip() without force (_command_update_transaction.py:551–558), so it never populates this recreation list. When user notes keep a supplemental skill directory alive, registration skips the now-markerless directory and the update succeeds without restoring its SKILL.md or registry entry. Pass the verified directories through the update/archive-install path as operation-scoped context, and add an update regression test that checks skill recreation and preservation of the notes.

AI-generated review for the requesting contributor by GitHub Copilot (model: unknown; automated review mode).

2 resolved since last review

Review effort: Balanced (adaptive)

@mnriem

mnriem commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

…skills

_unregister_extension_skills removed a non-generic supplemental skill
directory with shutil.rmtree, but ownership of such a directory is
established through the SKILL.md provenance marker alone, so any file a
user added next to it (notes, extra references) was deleted on
extension remove, and extension update runs the same removal. Remove
only the extension's own SKILL.md and then rmdir the directory once it
is empty -- the same per-file discipline the generic branch already
applies with its hash check.

Fixes github#5381

Assisted-by: Claude Code (model: Claude Fable 5, supervised)
…e by user files

Per-file removal leaves a skill directory without its SKILL.md when the
user's own files keep it alive, and registration then skipped it as an
unverifiable pre-existing directory, so a force reinstall (the update
path) silently dropped the skill. _unregister_extension_skills now
returns the directories whose ownership it verified before removing
their SKILL.md, remove() keeps them on the manager, and the force
reinstall hands them to _register_extension_skills as recreatable, so
the replacement recreates only the SKILL.md beside the user's files
without weakening the guard for unrelated user directories. Also adds
the supplemental-route symlink coverage: a user-retargeted SKILL.md
link survives removal, an expected dev-cache link outside the installed
directory is still retired.

Assisted-by: Claude Code (model: Claude Fable 5, supervised)
@aarjang
aarjang force-pushed the fix/5381-skill-removal-rmtree branch from 762e200 to 825328b Compare November 19, 2026 06:39
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 06:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

CLI updates and integration switches fail to preserve verified directory ownership, leaving supplemental skills without SKILL.md when their directories contain user notes.

2 open issues

Review effort: Balanced (adaptive)

Comment thread src/specify_cli/extensions/__init__.py Resolved
Comment thread src/specify_cli/extensions/__init__.py Resolved
…ration recreates the skill

The verified-ownership handoff added for the force reinstall lived in
manager state, so it missed the paths where removal and the next
registration are separate operations: the CLI extension update removes
and then installs without force, and switching the integration away
and back runs in separate processes. A skill directory kept alive by
the user's own files therefore stayed without its SKILL.md.

Removal now records such a directory in
.specify/extensions/.retained-skill-dirs.json (per extension,
project-relative, never outside the project), and registration consults
the record instead of weakening its guard, dropping the entry once the
directory carries its ownership marker again. One mechanism covers the
force reinstall, extension update, and the switch round trip, and the
recreatable_dirs parameter and manager attribute it replaces are gone.
An unrelated pre-existing user directory is still never written to.

Assisted-by: Claude Code (model: Claude Fable 5.2, supervised)
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 07:31
@aarjang

aarjang commented Nov 19, 2026

Copy link
Copy Markdown
Author

Addressed both findings in 27b09e6, by replacing the in-memory handoff rather than threading it through two more paths.

Both findings had the same cause: the verified-ownership handoff from the previous round lived in manager state, so it only reached the force reinstall. Removal and the next registration are separate operations on the CLI update path (remove(..., keep_config=True) then install_from_zip() without force, with the registry entry gone in between) and separate processes on a switch round trip.

  • One persistent record instead of a per-path handoff — when per-file removal leaves an ownership-verified skill directory alive (the user's files are in it), _unregister_extension_skills records it in .specify/extensions/.retained-skill-dirs.json, per extension and project-relative. _register_extension_skills consults that record in its pre-existing-directory guard and drops the entry once the directory carries its marker again. That covers the force reinstall, extension update, and the switch round trip with the same code, and the recreatable_dirs parameter and manager attribute from the previous round are removed.
  • Guard unchanged for unrelated directories — the record only flips the guard for the exact directory registration is about to write, for that extension. It fails closed: a malformed file authorizes nothing, an entry for another extension does not apply, and a directory outside the project is never recorded.
  • Regressions — test_cli_update_recreates_skill_beside_user_notes drives specify extension update through the CLI (the refreshed SKILL.md and the notes both survive, and the record is gone afterwards) and test_switch_round_trip_recreates_skill_and_keeps_user_dir_guard models the deactivate/reactivate lifecycle with a fresh manager and asserts an unrelated pre-existing user directory is still never written to. Both fail against the previous commit. test_retained_record_never_authorizes_an_unrecorded_directory pins the fail-closed cases.

Full suite on this branch: 17,010 passed, 620 skipped, 0 failed (macOS, .venv/bin/python -m pytest).

AI disclosure: This comment and the commit were drafted with Claude Code (model: Claude Fable 5.2), human-directed and reviewed, on behalf of @aarjang.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active path-validation flaw lets an unrecorded directory symlink inherit another directory’s ownership, potentially registering the wrong command’s SKILL.md and skipping creation of the intended skill.

1 open issue
2 resolved since last review

Review effort: Balanced (adaptive)

Comment on lines +2446 to +2453
try:
return (
skill_subdir.resolve()
.relative_to(self.project_root.resolve())
.as_posix()
)
except (OSError, ValueError):
return None
@mnriem

mnriem commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Removing supplemental extension skills deletes unrelated user files (rmtree)

3 participants